1. Field of the Invention
The invention relates to data storage and in particular to a system and method for recording and storing sequentially sensitive digital data such as digital media data so that it is physically contiguous and available for optimal retrieval speeds.
2. Related Art
Data storage technology has continued to improve throughout what has become known as the digital age. As a result, data storage devices have grown cheaper and faster with the passage of time. Data storage devices are still subject to some physical limitations, however, because devices such as hard drives still have moving parts. Currently, one of the most popular uses of the improvement in storage capacity and performance is the recording, editing and play back of digital media, including, but not limited to film, video, images, television, audio, and streaming media.
As media production has moved from film, video tape and audio tape to digital technology, fast and reliable storage of digital media has become vital. Digital media, like other digital data, may be stored in various ways including on magnetic tape, optical disks, or hard drives. In a digital media production environment, the hard drive is often utilized because of its combination of superior read and write access, large storage capacity, reliability, and speed.
The storage of digital media requires vast amounts of data storage capacity. This is especially true in digital media production where high resolution digital media is commonplace. For example, digital media for television and film typically have resolution ranging from 720×480 to 4096×3112 (i.e., 4 k) with frame rates ranging from 24 to 60 frames per second. However, emerging technologies, such as IMAX® theatres can have even greater resolution requirement. In addition, each color channel of the digital media may require between 8 and 10 bits of storage and up to 16 bits have been suggested.
It is known in the art that digital media may be compressed. Compression reduces the storage requirements, and consequently the data rate requirements for digital media. However, compression also adversely affects the quality of the digital media by introducing image artifacts or reducing detail. In digital media production environments, it is especially important to maintain the quality of digital media and thus uncompressed digital media, which preserves the original quality of the digital media, is preferred despite the additional resources it requires. The data storage requirements for uncompressed digital media are substantially greater than that of compressed digital media because the color, luminosity, and other aspects of every pixel are stored and maintained exactly.
The storage device used in recording, playing, or editing digital media must have sufficient throughput to read and write data in at least real time. If the throughput is insufficient then digital media frames can be dropped or skipped to the detriment of the overall quality of the digital media itself. In addition, editing may require even higher throughput than recording or play back because more than one digital media stream may be required such as to include special effects.
The throughput or data rate requirements for digital media are related to the storage requirements. For example, the compressed hour of digital media above at 720×480 would require a data rate of approximately 25 megabits per second and similar uncompressed digital media would require at data rate of 210 megabits per second. High definition digital media at 1920×1080 would respectively require 100 megabits per second or 1500 megabits per second for compressed and uncompressed digital media.
Thus, it can be seen that for both compressed and uncompressed digital media the data capacity and data rate for digital media storage devices is high and that these requirements are even higher in the realm of digital media editing or production.
It is known that reading fragmented or randomized data reduces disk throughput. Traditional systems have attempted to reduce data fragmentation, at least within individual files by executing defragmentation algorithms which attempt to identify individual files that have been subdivided with the divisions saved in discontinuous locations on a drive and move the divisions to an empty space large enough to store the entire file. Defragmentation is a complex and time consuming task especially given the file sizes and vast amounts of data required to store digital media. In fact, defragmentation of a disk can take several hours to several days, during which time the disk or array may have significantly reduced throughput or be completely unavailable for use. In addition, changing existing data (such as when editing digital media) or writing new data (such as when recording digital media) typically causes additional fragmentation and consequently slows the throughput of the drive and any array the drive is a part of.
Another situation that leads to randomizing and thus decreased performance occurs when multiple users (or even multiple programs) attempt to concurrently write a series of files to a shared storage device. Because there is no methodology currently in place to avoid randomizing, the two (or more) series of files written by concurrent users will almost invariably be interwoven with each other to at least some degree. The networked storage solutions currently available were specifically designed because multiple users often require access to the same data set. Thus, limiting access so there can be no concurrent use of such storage is rarely an acceptable option.
Thus, what is desired and disclosed herein is a system and method for recording and storing sequentially sensitive data in a physically contiguous manner on one or more disks.